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Foreword 


This  is  the  second  volume  of  a  technical  report  prepared  by  the  University  of  Dayton  Research 
Institute  for  the  Materials  and  Manufacturing  Directorate  (ML)  of  the  Air  Force  Research 
Laboratory  (AFRL),  Wright-Patterson  Air  Force  Base,  Ohio.  The  work  was  performed  under 
Contract  Number  F33615-95-C-5242  with  Mr.  Charles  F.  Buynak  (AFRL/MLLP)  as  the  Air 
Force  project  engineer.  The  technical  effort  was  performed  between  29  September  1995  and  31 
December  1999,  with  Dr.  Alan  P.  Berens  of  the  University  of  Dayton  Research  Institute  as  the 
principal  investigator. 

The  final  report  of  this  work  comprises  three  volumes.  Volume  1  presents  a  description  of 
changes  made  to  the  probability  of  detection  (POD)  analysis  program  of  Mil-HDBK-1823  and 
the  statistical  evaluation  of  modifications  that  were  made  to  version  3  of  the  Eddy  Current 
Inspection  System  (ECIS  v3).  Volume  2  contains  the  Users  Manual  for  the  version  3  update  of 
the  POD  program.  The  results  of  a  separate  study  for  predicting  POD  from  specimens  of  like 
geometry  and  materials  are  presented  in  Volume  3. 
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Section  1 

Introduction 


The  probability  of  detection  computer  program  POD  Version  3  is  a  program  that  interacts 
with  a  Microsoft  Excel  97  workbook  to  perform  a  POD  analysis  on  the  results  of  a 
nondestructive  evaluation  (NDE)  capability  demonstration.  The  interface  between  the  user  and 
POD  is  an  Excel  workbook.  The  POD  program  controls  the  analysis  though  a  window  that  opens 
and  closes  workbooks,  calls  for  recalculations  using  the  analysis  and  data  specified  on  the  input 
sheets  of  the  workbook,  and  executes  Excel  macros  that  produce  the  output  of  the  analysis.  POD 
names,  uses,  and  controls  a  maximum  of  1 1  sheets  of  the  workbook.  Two  of  these  sheets  contain 
the  input  information  for  an  analysis  and  the  other  nine  contain  analysis  output  in  the  form  of 
tables  and  graphs.  Graphical  output  sheets  are  optional. 

This  Users  Manual  assumes  familiarity  with  the  objectives  and  analysis  methods  for  fitting  the 
cumulative  lognormal  and  log  odds  models  to  d  versus  a  and  pass/fail  (find/no  find)  forms  of 
NDE  reliability  data.  See  References  1-4  for  details  of  such  analyses.  Reference  4  contains  a 
discussion  of  the  specific  analyses  performed  by  POD  Version  3. 

Section  2  presents  a  description  of  the  use  of  the  program  and  the  worksheets  of  a  POD 
workbook  for  both  the  d  versus  a  and  pass/fail  (find/no  find)  analyses.  The  descriptions 
emphasize  the  input  sheets  for  a  POD(a)  analysis  and  the  explanation  of  the  output  tables  and 
graphs.  The  input  and  output  sheets  are  standard  worksheets  of  an  Excel  workbook.  All  Excel  97 
commands  can  be  used  at  the  discretion  of  the  user.  Knowledge  of  Excel  is  assumed  and  no 
Excel  commands  are  discussed.  Section  3  presents  the  details  of  implementing  the  commands  of 
the  POD  window.  The  list  of  references  are  in  Section  4. 


1 


Section  2 

Description  of  POD  Version  3 


The  program  POD  Version  3  opens  an  Excel  workbook  and  uses  inspection  results  from  a  sheet 
named  Data  to  perform  analyses  as  specified  on  a  sheet  named  Info.  These  two  sheets  comprise 
the  input  to  the  POD  analysis.  The  output  is  produced  in  the  form  of  tables  and  graphs  on  as 
many  as  nine  new  workbook  sheets  that  are  regenerated  each  time  the  POD  maximum  likelihood 
calculations  are  executed.  If  the  data  in  a  workbook  have  not  been  previously  analyzed  by  POD, 
the  program  will  assist  users  in  setting  up  an  Info  sheet  and  will  immediately  perform  the  first  set 
of  calculations  and  create  a  Results  sheet  and  a  Residuals  sheet.  Pod  will  then  create  additional 
plots  upon  request.  If  a  valid  Info  sheet  is  present  when  POD  opens  the  workbook,  the  program 
will  recalculate  for  the  analysis  as  indicated  on  the  existing  Data  and  Info  sheets  and  will  renew 
all  existing  output  sheets.  This  recalculation  ensures  that  the  output  sheets  will  agree  with  the 
analysis  as  defined  by  the  current  Info  and  Data  sheets.  The  following  subsections  describe  the 
contents  of  the  input  and  output  sheets  of  POD  for  the  d  versus  a  and  pass/fail  analysis  methods. 
Material  that  is  common  to  both  analyses  is  repeated. 

2.1  d  versus  a  (Ahat)  Analysis 

The  input  for  a  POD  analysis  is  contained  on  the  Data  and  Info  sheets  of  the  Excel  workbook. 

The  Data  sheet  contains  the  flaw  sizes  and  inspection  responses,  d,  from  an  NDE  capability 
evaluation.  The  Info  sheet  contains  identification  and  modeling  information  that  will  be  used  by 
the  POD  program.  The  d  versus  a  analysis  output  comprises  as  many  as  nine  worksheets  that 
summarize  the  POD(a)  analysis  in  tables  and  graphs  The  specific  graphs  to  be  generated  are 
selected  by  the  analyst. 

When  POD  opens  a  new  Excel  worksheet  containing  d  inspection  results,  the  program  first 
ensures  that  the  sheet  is  named  Data.  An  Info  sheet  is  then  initiated  that  contains  the  minimum 
information  needed  to  perform  a  POD  analysis.  See  section  3.3.1  for  an  example.  Whenever 
POD  opens  a  workbook,  the  POD(a)  model  parameters  are  recalculated  to  ensure  that  all  results 
and  plots  were  generated  from  the  current  setup  on  the  Info  sheet.  After  a  workbook  has  been 
opened  and  recalculated,  the  Data  sheet  can  be  modified  and  the  Info  sheet  can  be  changed  to 
provide  a  more  complete  description  of  an  analysis  run  and  to  change  analysis  parameters. 

Columns  not  used  by-POD  on  the  Data  and  Info  worksheets  can  be  used  at  the  discretion  of  the 
analyst,  and  the  information  will  remain  in  the  workbook.  However,  POD  produces  output  on  a 
set  of  named  worksheets  and  these  output  sheets  are  recreated  every  time  the  analyses  are 
recalculated.  To  preserve  new  or  existing  worksheets,  they  must  be  assigned  names  that  are 
different  from  the  standard  POD  output  sheets. 

The  following  subsections  describe  the  contents  of  all  of  the  worksheets  of  an  d  versus  a 
analysis. 
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2.1.1  Data  Sheet  for  Ahat  Analysis 


The  inspection  result  input  to  POD  is  contained  on  the  Data  sheet  of  the  workbook.  The  columns 
of  the  Data  sheet  contain  crack  identification  fields,  crack  size  fields  and  the  inspection  results 
obtained  for  each  of  crack  sites.  The  first  row  of  the  spreadsheet  must  be  a  header  row 
containing,  as  a  minimum,  the  names  of  the  crack  size  column  and  inspection  columns  to  be  used 
in  the  analysis.  Each  succeeding  row  after  the  first  contains  the  identifying  information,  sizes  of 
the  cracks,  and  all  d  inspection  results  for  each  crack  of  the  specimen  set.  The  POD  program  uses 
only  the  column  that  contains  the  crack  size  and  the  columns  that  contain  the  d  values.  The  POD 
program  will  request  the  column  letter  that  contains  the  results  of  the  first  inspection  of  each 
crack.  Results  from  any  additional  inspections  must  be  in  columns  that  are  contiguous  to  the  first 
set.  If  other  information  is  placed  on  the  Data  sheet,  it  must  be  separated  by  at  least  one  column 
from  the  d  values. 

As  an  example.  Figure  1  presents  part  of  a  spreadsheet  that  contains  the  first  15  cracks  from  a 
Data  sheet  for  an  d  versus  a  analysis.  The  first  three  columns  of  this  example  Data  sheet  contain 
identifying  information  for  the  cracks  and  these  columns  will  be  ignored  by  POD.  Both  crack 
length  and  depth  are  listed  for  the  cracks,  and  the  analyst  will  specify  which  is  to  be  used  in  the 
analysis  on  the  Info  sheet.  Each  crack  shown  in  the  example  data  sheet  has  been  inspected  twice 
and  the  inspection  designations  are  Insl  and  Ins  2.  Note  that  some  of  the  inspection  sites  do  not 
contain  flaws  and  this  absence  of  flaws  is  indicated  by  “blank”  in  the  size  columns.  These 
inspection  sites  were  the  unflawed  sites,  and  such  sites  will  have  no  effect  on  the  analysis. 
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Figure  1.  Example  Data  Worksheet  for  an  Ahat  Analysis  Using  POD 

The  data  to  be  used  in  an  analysis  are  selected  on  the  Data  worksheet.  The  POD  program  will  use 
all  cracks  that  have  a  clear  (uncolored)  background.  To  eliminate  any  cracks  or  range  of  cracks 
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from  an  analysis,  select  them  and  add  a  highlight  background  color.  This  selection  process  makes 
it  very  convenient  to  select  particular  inspections  of  all  cracks,  to  exclude  ranges  of  crack  size, 
and  to  check  the  influence  of  apparent  outliers.  Note  than  an  asterisk  in  the  inspection  result 
column  indicates  that  the  site  was  not  inspected.  A  missing  inspection  is  distinct  from  a  missed 
flaw  and  the  program  will  not  interpret  the  asterisk  as  a  miss. 

Note  that  POD  provides  an  easy  sort  by  crack  size  of  the  data  in  the  worksheet  (see  POD 
window.  Tasks  -  Sort  by  size,  section  3.4).  But  note  also  that  the  Data  worksheet  is  an  Excel 
spreadsheet,  and  any  spreadsheet  functions  can  be  directly  performed.  It  is  necessary  to  maintain 
the  inspection  results  in  contiguous  columns  starting  in  the  column  as  indicated  on  the  Info 
worksheet. 

2.1.2  Info  Sheet  for  Ahat  Analysis 

The  Info  sheet  of  a  POD  workbook  contains  information  that  the  program  will  use  to  generate  the 
output  tables  and  graphs.  An  example  Info  Sheet  for  an  d  versus  a  analysis  is  shown  in  Figure  2. 
The  information  is  grouped  in  4  categories  for  ease  of  interpretation.  Entries  in  column  A  define 
the  type  of  information  in  the  row  but  only  the  entries  that  end  with  a  colon  have  meaning  to 
POD.  For  example,  the  names  of  the  4  categories  of  information  are  ignored  by  POD  as  would  be 
any  other  entry.  Further,  the  order  of  the  lines  on  the  Info  sheet  is  not  important.  However,  exact 
spelling  of  the  entries  that  end  with  a  colon  must  be  maintained. 

Column  B  of  the  Info  sheet  contains  the  following  information  for  the  POD  pertinent  rows: 

•  Title  -  Identification  entry  that  will  appear  on  the  Results  output  sheet  and  all  graphs. 

•  Subtitle  -  Additional  identification.  The  first  four  Subtitle  entries  will  appear  on  all  graphs. 
All  subtitle  entries  will  appear  on  the  Results  sheet  in  the  order  in  which  they  are  on  the  Info 
sheet.  As  many  subtitle  lines  as  desired  can  be  inserted  in  the  Info  sheet  by  adding  rows  at  the 
bottom  with  “Subtitle:”  in  column  A.  All  Subtitle  lines  will  be  printed  on  the  Results  sheet. 

•  Flaw  Column  -  Column  letter  of  the  Data  sheet  containing  the  flaw  sizes  to  be  used  in  the 
analysis. 

•  Flaw  Units  -  Flaw  size  units  that  will  appear  in  the  output. 

•  Flaw  Transform  -  Definition  of  flaw  size  transformation  to  be  used  in  the  analysis.  The 
default  transformation  for  flaw  size  is  the  natural  logarithm.  Different  transformations  can  be 
used  by  entering  the  transformation  name  in  the  row  labeled  Flaw  Transform.  Column  B 
should  contain  one  of  the  following  entries: 

Column  B  Entry:  Transform: 

none  y  =  x 

log  y  =  ln(x) 

inverse  y  =  llx 

custom  User  defined 

Either  no  entry  or  log  in  column  B  is  the  default  natural  logarithm  transformation.  In  a 
custom  transformation.  Column  D  in  the  same  row  must  contain  a  formula  transforming  the 
value  in  column  C  of  the  same  row.  Similarly,  column  F  must  contain  the  formula  inverting 
the  transform  for  the  value  in  column  E  of  the  same  row.  Figure  3  shows  an  example  of  the 
formulas  needed  to  define  a  log  transformation  using  the  custom  option. 
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Figure  2.  Example  Info  Sheet  for  an  Ahat  Analysis  Using  POD 
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Figure  3.  Example  Showing  Eormulas  for  Custom  Data  Transformations 
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•  Insp  Start  -  Data  sheet  column  that  contain  the  first  set  of  inspection  results. 

•  Insp  Transform  -  Definition  of  flaw  size  transformation  to  be  used  in  the  analysis.  The 
default  transformations  for  signal  response,  a,  is  the  natural  logarithm.  Different 
transformations  can  be  used  by  entering  the  transformation  name  in  the  row  labeled  Insp 
Transform.  Column  B  should  contain  one  of  the  following  entries: 

Column  B  Entry:  Transform: 

none  y  =  .r 

log  y  =  ln(.r) 

inverse  y  =  1/x 

custom  User  defined 

Either  no  entry  or  log  in  column  B  is  the  default  natural  logarithm  transformation.  In  a 
custom  transformation,  Column  D  in  the  same  row  must  contain  a  formula  transforming  the 
value  in  column  C  of  the  same  row.  Similarly,  column  F  must  contain  the  formula  inverting 
the  transform  for  the  value  in  column  E  of  the  same  row.  Figure  3  shows  an  example  of  the 
formulas  needed  to  define  a  log  transformation  using  the  custom  option. 

•  Signal  Min  -  Minimum  a  value  that  can  be  registered  by  the  NDE  system. 

•  Signal  Max  -  Maximum  a  value  that  can  be  registered  by  the  NDE  system. 

•  Analysis  -  Ahat  directs  POD  to  perform  an  a  versus  a  analysis. 

•  Version  -  Version  of  POD  being  used  for  the  analysis. 

•  Thresholds  -  Columns  B  and  C  define  the  range  of  a  decision  thresholds  for  the  ago  versus 
ddec  plot.  Threshold  entries  in  column  D,  E,  etc.  can  be  used  to  extend  or  refine  the  ago  versus 

^dec  plot. 

•  POD  Threshold  -  a  decision  threshold  for  a  plot  of  the  POD(a)  function  with  confidence 
bound. 

•  POD  level  -  POD  percent  value  for  which  confidence  bound  will  be  calculated.  Default  value 
is  90  percent. 

•  Confidence  -  Percent  confidence  bound  for  POD  value.  Choices  are  90,  95,  and  99.  Default 
value  is  95. 

2.1.3  Results  Sheet  from  Ahat  Analysis 

The  Results  sheet  contains  a  summary  of  the  a  versus  a  analysis.  An  example  is  presented  as 
Figure  4.  The  top  lines  repeat  all  of  the  identifying  information  from  the  Title  and  Subtitle  lines 
of  the  Info  sheet.  The  identification  lines  are  followed  by  the  range  of  crack  sizes,  the  number  of 
cracks,  the  number  of  censored  recordings,  and  the  inspection  titles.  In  the  example  sheet,  there 
are  two  inspections  per  crack  identified  as  Ins  I  and  Ins  2,  the  inspection  names  from  the  Data 
sheet.  Four  of  the  cracks  had  both  responses  less  than  the  minimum  (signal  threshold),  and  one 
crack  had  one  response  above  and  one  below  the  minimum.  None  of  the  cracks  had  a  values 
above  the  maximum  (saturation). 

The  analysis  type  and  model  for  the  analysis  are  followed  by  the  parameter  estimates  and 
standard  errors  of  the  a  versus  a  fit  parameters.  The  standard  errors  (Std.  Error)  are  the  standard 
deviations  of  the  estimates  of  the  parameters  and  indicate  the  degree  of  precision  of  the 
estimates.  The  Residual  Error  is  the  standard  deviation  of  the  differences  between  the  average  a 
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values  and  the  linear  fit.  The  Repeatability  Error  is  the  pooled  standard  deviation  of  the  repeated 
d  values  for  each  crack.  The  results  of  the  hypothesis  tests  for  model  fit  are  presented  in  terms  of 
the  calculated  test  statistic  and  the  significance  level  of  the  test.  High  values  of  P  indicate  that  the 
data  are  compatible  with  the  assumption. 
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Figure  4.  Example  POD  Results  Sheet  from  Ahat  Analysis 

The  POD  parameters  are  summarized  only  in  terms  of  Sigma  and  aso,  ago,  and  ago/gs  values  for 
the  POD  Threshold  on  the  Info  sheet.  The  parameter  fa  depends  on  the  decision  threshold,  ^  =  In 
(ajo).  The  variance-covariance  matrix  for  the  estimates  of  fj.  and  (Tare  contained  in  a  different 
worksheet. 
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2.1.4  Residuals  Sheet  from  Ahat  Analysis 


Figure  5  presents  a  portion  of  the  Residuals  sheet  of  the  Ahat  example.  The  Residuals  sheet  for 
an  a  versus  a  analysis  comprises  four  tables  that  are  used  to  generate  the  Ahat  vs.  a,  Fit  Plot,  and 
Residuals  Plot  graphs.  The  residual  table  lists  only  the  cracks  used  in  the  analysis  and  includes 
the  crack  sizes,  the  average  d  for  each  crack,  the  log  of  crack  size,  log  of  average  d,  and  the 
differences  (residuals)  between  the  average  and  predicted  log  d  values.  The  d  versus  a  table  lists 
all  of  the  cracks  from  the  Data  sheet  and  contains  the  crack  sizes,  the  individual  d  values  from  all 
inspections,  and  the  predicted  d  for  the  crack  size.  The  fit  table  is  a  two  cell  by  two  cell  table  of 
the  straight  line  fit  to  the  data  used  in  the  analysis.  The  min/max  table  is  used  to  plot  the 
minimum  and  maximum  d  value  over  the  range  of  data. 
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Figure  5.  Example  Residuals  Sheet  from  Ahat  Analysis 
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2.1.5  Ahat  vs.  a  Sheet  from  Ahat  Analysis 


The  Ahat  vs.  a  sheet  contains  a  plot  of  d  versus  a  for  all  of  the  inspections  of  all  of  the  cracks.  An 
example  of  this  plot  for  the  data  from  Figure  5  is  presented  in  Figure  6.  This  plot  aids  in  the 
selection  of  the  range  of  crack  sizes  or  in  the  model  formulation  to  be  used  in  the  POD(a) 
analysis.  It  also  can  identify  individual  inspection  results  that  do  not  agree  with  other  inspections 
of  the  same  crack. 


TManilafiSrtidNBiin  MCiltrS 
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Figure  6.  Example  Ahat  vs.  a  Sheet  from  Ahat  Analysis 


2.1.6  Fit  Plot  Sheet  from  Ahat  Analysis 

The  Fit  Plot  sheet  for  an  d  versus  a  analysis  contains  a  plot  of  the  average  d  for  each  crack  versus 
the  crack  size  with  a  superimposed  straight  line  fit  obtained  from  the  analysis.  Figure  7  presents 
an  example  fit  plot  for  the  data  of  Figures  3,  4  and  5.  The  Fit  plot  provides  for  easy  visual 
inspection  of  the  goodness  of  fit  and  can  be  used  to  choose  crack  size  regions  for  which  the 
relation  might  be  more  linear. 

2.1.7  Residuals  Plot  Sheet  from  Ahat  Analysis 

The  Residuals  Plot  presents  the  difference  between  average  and  predicted  log  d  as  a  function  of 
the  size  of  the  crack.  An  example  Residuals  Plot  is  presented  in  Figure  8  for  the  fit  of  Figure  7. 
The  residuals  plot  for  d  versus  a  data  aids  in  identifying  crack  size  regions  for  which  the  fit  may 
not  be  linear  or  for  which  the  scatter  in  residuals  is  changing.  The  Residuals  Plot  is  also  useful  in 
identifying  the  outlying  data  points  that  may  be  affecting  the  tests  of  hypotheses. 
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Figure  7.  Example  Fit  Plot  Sheet  from  Ahat  Analysis 
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Figure  8.  Example  Residuals  Plot  Sheet  from  Ahat  Analysis 
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2.1.8  Threshold  Data  Sheet  from  Ahat  Analysis 


The  Threshold  Data  sheet  is  a  table  that  contains  aso,  ci90,  CI90/95,  Vu,  Vn,  and  V22  for  ranges  of 
thresholds  that  are  specified  for  the  analysis  on  the  Info  page.  The  program  will  insert  nine  equally 
spaced  threshold  values  between  the  listed  thresholds.  More  than  one  threshold  range  can  be 
included  by  entering  larger  threshold  values  in  contiguous  columns  in  the  POD  Threshold  row  of 
the  Info  sheet.  Figure  9  presents  the  Threshold  Data  sheet  for  the  analysis  of  the  data  in  Figure  6. 
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Figure  9.  Example  Threshold  Data  Sheet  from  Ahat  Analysis 


2.1.9  Threshold  Plot  Sheet  from  Ahat  Analysis 

The  Threshold  Plot  presents  the  estimates  of  ago  and  agom  as  functions  of  the  decision  threshold. 
The  Threshold  Plot  for  the  data  of  the  Figures  6  and  9  is  presented  in  Figure  10.  This  plot  has 
become  the  most  useful  characterization  of  NDE  capability  for  d  versus  a  data  because  a  single 
demonstration  of  capability  is  often  used  for  different  target  ago  values.  Eurther,  thresholds  in 
automated  systems  often  need  to  be  adjusted,  and  the  threshold  plots  readily  yield  the  ago  values 
that  would  result  for  different  choices. 

2.1.10  POD  Data  Sheet  from  Ahat  Analysis 

The  POD  Data  sheet  comprises  three  columns  that  contain  the  array  of  crack  sizes,  the  estimated 
POD(a)  function,  and  the  confidence  bound  for  POD(a).  Eigure  1 1  presents  a  small  portion  of  the 
POD  Data  Sheet  for  the  example  a  versus  a  analysis. 
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Figure  10.  Example  Threshold  Plot  Sheet  from  Ahat  Analysis 
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Figure  11.  Example  POD  Data  Sheet  from  Ahat  Analysis 
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2.1.11  POD  Sheet  from  Ahat  Analysis 

The  POD  sheet  contains  the  POD(a)  function  and  its  confidence  bound.  Figure  12  presents  the 
plot  of  the  POD  sheet  for  the  example  d  versus  a  analysis.  In  an  d  versus  a  data  analysis,  the 
decision  threshold  of  the  POD(a)  function  is  that  identified  as  the  POD  Threshold  on  the  Info 
sheet.  In  Figure  12,  the  POD(a)  function  and  the  confidence  bound  stop  at  the  maximum  crack 
size  (a  =  17  mil)  in  the  data  of  the  analysis. 


IriMilum -Small  BOH  htolss 

Exiaivle  faiE' a 
tais  4 
E>!4in(ile' Hub  S 


Figure  12.  Example  POD  Plot  with  Confidence  Bound  from  Ahat  Analysis 
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2.2  Pass/Fail  Analysis 


The  input  for  a  POD  analysis  is  contained  on  the  Data  and  Info  sheets  of  the  Excel  workbook. 

The  Data  sheet  contains  the  flaw  sizes  and  the  quantified  find/no  find  decisions  (one  for  a  find 
and  zero  for  a  no  find)  from  an  NDE  capability  evaluation.  The  Info  sheet  contains  identification 
and  modeling  information  that  will  be  used  by  the  POD  program.  The  pass/fail  analysis  output 
comprises  as  many  as  six  worksheets  that  summarize  the  POD(a)  analysis  in  tables  and  graphs 
The  specific  graphs  to  be  generated  are  selected  by  the  analyst. 

When  POD  opens  a  new  Excel  worksheet  containing  the  pass/fail  inspection  results,  the  program 
first  ensures  that  the  sheet  is  named  Data.  An  Info  sheet  is  then  initiated  that  contains  the 
minimum  information  needed  to  perform  a  POD  analysis.  See  section  3.3.1  for  an  example. 
Whenever  POD  opens  a  workbook,  the  POD(a)  model  parameters  are  recalculated  to  ensure  that 
all  results  and  plots  were  generated  from  the  current  setup  on  the  Info  sheet.  After  a  workbook 
has  been  opened  and  recalculated,  the  Data  sheet  can  be  modified  and  the  Info  sheet  can  be 
changed  to  provide  a  more  complete  description  of  an  analysis  run  and  to  change  analysis 
parameters. 

Columns  not  used  by  POD  on  the  Data  and  Info  worksheets  can  be  used  at  the  discretion  of  the 
analyst,  and  the  information  will  remain  in  the  workbook.  However,  POD  produces  output  on  a 
set  of  named  worksheets  and  these  output  sheets  are  recreated  every  time  the  analyses  are 
recalculated.  To  preserve  new  or  existing  worksheets,  they  must  be  assigned  names  that  are 
different  from  the  standard  POD  output  sheets. 

The  following  subsections  describe  the  contents  of  all  of  the  worksheets  of  a  pass/fail  analysis. 

2.2.1  Data  Sheet  for  Pass/Fail  Analysis 

The  inspection  result  input  to  POD  is  contained  on  the  Data  sheet  of  the  workbook.  The  columns 
of  the  Data  sheet  contain  crack  identification  fields,  crack  size  fields  and  the  inspection  results 
(one  for  a  find  and  zero  for  a  miss)  that  were  obtained  from  the  inspections  of  the  crack  sites.  The 
first  row  of  the  spreadsheet  must  be  a  header  row  containing,  as  a  minimum,  the  names  of  the 
crack  size  column  and  inspection  columns  to  be  used  in  the  analysis.  Each  succeeding  row  after 
the  first  contains  identifying  information  (if  any),  sizes  of  the  cracks,  and  the  inspection  results 
for  each  crack  of  the  specimen  set.  The  POD  program  uses  only  the  columns  that  contain  the 
crack  size  and  the  inspection  results.  The  POD  program  will  request  the  column  letter  that 
contains  the  results  of  the  first  inspection  of  each  crack.  Results  from  any  additional  inspections 
must  be  in  columns  that  are  contiguous  to  the  first  set.  If  other  information  is  placed  on  the  Data 
sheet,  it  must  be  separated  by  at  least  one  column  from  the  inspection  results  values. 

As  an  example,  Eigure  13  presents  part  of  a  spreadsheet  that  contains  the  first  17  cracks  from  a 
Data  sheet  for  a  pass/fail  analysis.  The  first  column  of  this  example  Data  sheet  contains 
identifying  information  for  the  cracks  and  this  column  will  be  ignored  by  POD.  The  crack  size 
information  is  in  the  column  named  Depth.  The  analyst  will  specify  this  column  on  the  Info  sheet 
as  the  Flaw  Column  to  be  used  in  the  analysis.  Each  crack  shown  in  the  example  data  sheet  was 
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inspected  once  and  the  inspection  designation  is  Ins  1 .  More  inspections  for  each  crack  are 
possible.  Note  that  inspection  sites  that  do  not  contain  flaws  can  be  included  on  the  Data  sheet. 
This  absence  of  a  flaw  at  a  site  can  be  indicated  by  an  initial  alpha  character  such  as  “blank”  in 
the  size  column.  Such  inspection  sites  will  be  ignored  in  the  analysis.  Note  also  than  an  asterisk 
in  the  inspection  result  column  indicates  that  the  site  was  not  inspected.  A  missing  inspection  is 
distinct  from  a  missed  flaw  and  the  program  will  not  interpret  the  asterisk  as  a  miss. 


Figure  13.  Example  Data  Worksheet  for  a  Pass/Fail  Analysis  Using  POD 


2.2.2  Info  Sheet  for  Pass/Fail  Analysis 

The  Info  sheet  of  a  POD  workbook  contains  information  that  the  program  will  use  to  generate  the 
output  tables  and  graphs.  An  example  Info  Sheet  for  a  pass/fail  analysis  is  shown  in  Figure  14. 
The  information  is  grouped  in  4  categories  for  ease  of  interpretation.  Entries  in  column  A  define 
the  type  of  information  in  the  row  but  only  the  entries  that  end  with  a  colon  have  meaning  to 
POD.  For  example,  the  names  of  the  4  categories  of  information  are  ignored  by  POD  as  would  be 
any  other  entry.  Further,  the  order  of  the  lines  on  the  Info  sheet  is  not  important.  However,  exact 
spelling  of  the  entries  that  end  with  a  colon  must  be  maintained. 
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Figure  14.  Example  Info  Sheet  for  a  Pass/Fail  Analysis  using  POD 

Column  B  of  the  Info  sheet  contains  the  following  information  for  the  POD  pertinent  rows: 

•  Title  -  Identification  entry  that  will  appear  on  the  Results  output  sheet  and  all  graphs. 

•  Subtitle  -  Additional  identification.  The  first  four  Subtitle  entries  will  appear  on  all  graphs. 
All  subtitle  entries  will  appear  on  the  Results  sheet  in  the  order  in  which  they  are  on  the  Info 
sheet.  As  many  subtitle  lines  as  desired  can  be  inserted  in  the  Info  sheet  by  adding  rows  at  the 
bottom  with  “Subtitle:”  in  column  A.  All  Subtitle  lines  will  be  printed  on  the  Results  sheet. 

•  Flaw  Column  -  Column  letter  of  the  Data  sheet  containing  the  flaw  sizes  to  be  used  in  the 
analysis. 

•  Flaw  Units  -  Flaw  size  units  that  will  appear  in  the  output. 

•  Flaw  Transform  -  Definition  of  flaw  size  transformation  to  be  used  in  the  analysis.  The 
default  transformation  for  flaw  size  is  the  natural  logarithm.  Different  transformations  can  be 
used  by  entering  the  transformation  name  in  the  row  labeled  Flaw  Transform.  Column  B 
should  contain  one  of  the  following  entries: 
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Transform: 


Column  B  Entry: 

none  j  ^ 

log  j  =  ln(^) 

inverse  j  =  Hx 

custom  User  defined 

Either  no  entry  or  log  in  column  B  is  the  default  natural  logarithm  transformation.  In  a 
custom  transformation,  Column  D  in  the  same  row  must  contain  a  formula  transforming  the 
value  in  column  C  of  the  same  row.  Similarly,  column  F  must  contain  the  formula  inverting 
the  transform  for  the  value  in  column  E  of  the  same  row.  Figure  15  shows  an  example  of  the 
formulas  needed  to  define  a  log  transformation  using  the  custom  option. 
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FLAW  INFO 
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Flaw  Column; 
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11 

Flaw  Units: 

mil 

12 

Flaw  Transform: 

custom 

10  =LN(C12) 

2.303  =EXP(E12) 

Figure  15.  Example  Showing  Formulas  for  Custom  Data  Transformation  of  Flaw  Size 

•  Insp  Start  -  Data  sheet  column  that  contain  the  first  set  of  inspection  results. 

•  Analysis  -  Pass/Fail  directs  POD  to  perform  a  pass/fail  analysis. 

•  Version  -  Version  of  POD  being  used  for  the  analysis. 

•  Amax  -  Upper  limit  of  crack  size  on  plots  if  POD  defaults  are  not  wanted. 

•  Guess  -  Columns  B  and  C  can  contain  initial  estimates  of  |l  and  o,  respectively,  that  will 
override  the  POD  default  initial  estimates  in  the  iterative  m  estimation  calculations.  See 
section  2.2.5. 

•  POD  level  -  POD  percent  value  for  which  confidence  bound  will  be  calculated.  Default  value 
is  90  percent. 

•  Confidence  -  Percent  confidence  bound  for  POD  value.  Choices  are  90,  95,  and  99.  Default 
value  is  95. 

2.2.3  Results  Sheet  from  Pass/Fail  Analysis 

A  Results  sheet  for  the  example  pass/fail  analysis  is  presented  in  Figure  16.  The  top  lines  repeat 
all  of  the  identifying  information  from  the  Title  and  Subtitle  lines  of  the  Info  sheet.  Next  are  the 
range  of  crack  sizes  in  the  analyzed  data,  the  number  of  unique  crack  sizes,  and  the  total  number 
of  cracks  in  the  analysis.  This  example  has  one  inspection  that  is  named  Ins  I.  The  analysis  and 
model  are  identified  and  followed  by  the  estimates  of  the  POD(a)  parameters.  Estimates  of  a 50, 
ago,  and  ^90/95  are  listed  along  with  the  variance-covariance  matrix  of  the  parameter  estimates. 

2.2.4  Residuals  Sheet  from  Pass/Fail  Analysis 

The  Residuals  sheet  for  a  pass/fail  analysis  comprises  the  observed  proportion  of  the  cracks  of 
each  size  that  were  detected,  the  POD(a)  estimate  for  each  crack  size,  and  the  difference  between 
the  observed  and  estimated  POD.  Figure  17  presents  the  small  crack  portion  of  the  Residuals 
sheet  for  the  pass/fail  example. 
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Figure  16.  Example  POD  Results  Sheet  from  Pass/Fail  Analysis 
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Figure  17.  Example  Residuals  Sheet  from  Example  Pass/Fail 
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2.2.5  Solve  Sheet  from  Pass/Fail  Analysis 


The  pass/fail  Solve  sheet  tracks  the  iterations  of  the  parameters,  usually  to  convergence  for 
reasonably  designed  capability  demonstrations.  If  convergence  is  not  reached  in  20  iterations, 
POD  changes  the  initial  estimates  and  tries  again.  If  convergence  is  not  reached  at  the  end  of 
pod’s  pattern  of  changes,  the  program  returns  a  message  that  convergence  could  not  be  reached. 
If  the  analyst  so  chooses,  initial  guesses  can  be  entered  on  the  Info  sheet  in  the  row  labeled  Guess 
and  the  manually  entered  values  will  override  POD’s  initial  guesses.  The  Solve  sheet  on  occasion 
has  indicated  the  direction  of  changes  that  should  be  made  to  the  initial  estimates.  Note  that 
convergence  to  a  maximum  likelihood  solution  is  not  always  possible.  The  Solve  sheet  for  the 
pass/fail  example  is  shown  in  Figure  15.  Convergence  was  reached  in  6  iterations  in  this 
example. 
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Figure  18.  Example  Solve  Sheet  from  Example  Pass/Eail  Analysis 


2.2.6  Fit  Plot  Sheet  from  Pass/Fail  Analysis 

The  Fit  Plot  sheet  for  a  pass/fail  analysis  is  a  plot  showing  the  POD(a)  fit  on  the  observed 
detection  probabilities.  Observed  detection  probability  for  a  crack  size  is  the  proportion  of  all 
inspections  of  cracks  of  that  size  that  resulted  in  detections.  Figure  19  is  the  Fit  Plot  for  the 
example  pass/fail  analysis.  A  subjective  judgement  of  goodness  of  fit  can  be  made  from  this  plot. 

2.2.7  POD  Data  Sheet  from  Pass/Fail  Analysis 

The  POD  Data  sheet  comprises  three  columns  that  contain  an  array  of  crack  sizes,  the  estimated 
POD(a)  function,  and  the  confidence  bound  for  POD(a).  Figure  20  presents  a  small  portion  of  the 
POD  Data  sheet  for  the  example  pass/fail  analysis. 
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Figure  19.  Example  Fit  Plot  Sheet  from  Pass/Fail  Analysis 
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Figure  20.  Example  POD  Data  Sheet  from  Pass/Fail  Analysis 
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2.2.8  POD  Sheet  from  Pass/Fail  Analysis 

The  POD  sheet  contains  plots  of  the  POD(a)  function  and  the  confidence  bound.  Figure  21 
presents  the  POD  sheet  for  the  example  pass/fail  analysis. 


Exainpl«  iBi*  S 
Exjtrnpl*  I>i4  9 
EX*mpl«  IKI4  4 
E]cunpl«  iBi*  E 


Figure  21.  Example  POD  Plot  with  Confidence  Bound  from  Example  Pass/Fail  Analysis 
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Section  3 

Implementation  of  POD  Version  3 


This  section  describes  how  to  use  POD  Version  3  to  perform  POD  analyses  of  d  versus  a  and 
pass/fail  data  that  are  contained  in  an  Excel  workbook. 

3.1  POD  Version  3.0  Installation 

To  install,  copy  POD  Version  3  to  any  directory  in  the  search  path.  The  POD  Version  3  requires 
Microsoft  Windows  95  (or  later)  and  Excel  97  (or  later).  The  POD  program  requires  design  link 
libraries  MPC42.DEE,  dated  04/27/1999  or  later,  and  MSVCRT.DEE,  dated  07/23/1996,  or  later. 

3.2  Starting  POD 

Start  POD  by  double  clicking  on  the  POD  icon.  A  shortcut  to  the  icon  can  be  placed  on  the 
screen,  if  desired.  When  POD  activates,  it  first  brings  up  a  copy  of  Excel  and  opens  the  POD 
window,  Eigure  22.  Do  not  close  this  copy  of  Excel  while  POD  is  running. 


Figure  22.  POD  Window 


3.3  POD  File  Operations 

The  File  menu  in  the  POD  window  contains  items  associated  with  opening  and  saving  the  Excel 
data  and  analysis  files.  It  also  contains  a  list  of  the  most  recently  used  files  and  a  menu  item  for 
exiting  the  application.  The  POD  File  menu  is  shown  in  Eigure  23. 
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Figure  23.  POD  File  Menu 


3.3.1  Opening  a  File 

Selecting  the  File  -  New  menu  item  opens  the  Excel  workbook  to  a  blank  Data  sheet  for  direct 
entry  of  new  data.  A  message  appears  that  prompts  the  user  to  enter  the  data  and  Recalculate. 

Data  in  the  correct  format  for  a  Data  sheet  can  be  entered  by  any  convenient  method.  See 
subsection  3.4  for  the  Recalculation  command.  In  this  File  -  New  sequence,  the  Recalculation 
command  will  first  open  an  Info  Worksheet  window  that  requests  the  necessary  information  to 
start  an  analysis  (as  discussed  below).  POD  will  then  generate  a  preliminary  Info  sheet  and  will 
perform  the  analysis  as  indicated  on  the  Info  and  Data  sheets.  Note  that  is  often  more  convenient 
to  enter  the  data  in  a  spreadsheet  without  opening  POD.  The  file  can  then  be  saved  and  opened 
using  the  File  -  Open  command. 

Selecting  the  File  -  Open  command  opens  an  Excel  workbook  that  contains  a  valid  Data  sheet.  If 
the  workbook  also  contains  a  valid  Info  sheet,  POD  will  automatically  recalculate  the  analysis  as 
indicated  on  the  Info  sheet  using  the  selected  data  on  the  Data  sheet.  This  would  be  the  case  for 
files  that  had  been  previously  opened  by  POD.  If  the  workbook  does  not  contain  an  Info  sheet, 
POD  opens  an  Info  Worksheet  window  that  prompts  for  an  initial  set  of  information  and  creates  a 
preliminary  Info  sheet.  Eigure  24  presents  the  Info  Worksheet  window  for  creating  the  preliminary 
Info  sheet.  If  the  analysis,  flaw  column,  or  flaw  inspection  entries  are  not  entered  correctly  on  the 
Info  sheet,  POD  may  not  be  able  to  perform  the  analysis. 
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Figure  24.  POD  Info  Worksheet 

3.3.2  Save  and  Save  As  Menu  Items 

The  File  -  Save  and  File  -  Save  As  menu  items  operate  as  in  Excel. 

3.3.3  Recently  Used  File  List 

Double  clicking  on  a  file  in  the  list  of  recently  used  files  will  open  the  workbook. 

3.3.4  Exit  Menu  Item 

The  POD  File  -  Exit  menu  item  will  first  ask  if  the  workbook  should  be  saved  since  it  is 
automatically  modified  upon  opening.  The  POD  program  will  then  close  both  the  POD  program 
and  the  Excel  application.  The  workbook  can  also  be  closed  using  the  Excel  File  commands.  It  is 
not  desirable  to  close  the  Excel  application  opened  by  POD  using  the  Excel  close  commands. 
Closing  Excel  early  can  cause  POD  to  generate  communication  errors,  since  POD  expects  the 
workbook  it  created  to  be  open.  If  closing  Excel  early  produces  one  of  a  variety  of  warning  and 
error  messages,  close  and  restart  POD. 

3.4  POD  Tasks  Menu 

There  are  two  menu  items  under  the  Tasks  menu  (Eigure  25).  Recalculate  uses  the  information 
on  the  Info  sheet  and  the  selected  data  from  the  Data  sheet  to  calculate  the  maximum  likelihood 
estimates  of  the  parameters  of  the  model  and  to  regenerate  and  replace  all  existing  POD  tables 
and  graphs.  Recalculating  does  not  change  the  Info  and  Data  sheets  or  any  sheet  with  a  name  not 
used  by  POD. 
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Figure  25.  POD  Recalculate  Menu 


It  is  often  necessary  or  convenient  to  select  ranges  of  crack  sizes  for  analysis.  The  Tasks  - 
Sort  by  size  menu  item  rearranges  the  contiguous  inspection  result  columns  of  the  Data  sheet  in 
ascending  order  of  the  flaw  size  column. 

3.5  POD  Charts  Menu 

The  Excel  macros  that  generate  the  graphs  are  not  activated  the  first  time  POD  opens  a 
workbook.  The  Charts  menu  allows  the  user  to  choose  the  particular  graphs  of  interest. 
Computing  time  is  saved  when  unwanted  graphs  are  not  generated.  The  Charts  menu  is  shown  in 
Figure  26.  Clicking  on  a  specific  graph  activates  the  macro  that  produces  the  graph  as  a  sheet  in 
the  workbook.  If  a  graph  sheet  exists  in  the  workbook  when  a  recalculation  is  performed,  a  new 
graph  will  overwrite  the  existing  graph.  Changing  the  name  of  a  graph  (or  any  other  worksheet) 
generated  by  POD  to  a  name  not  used  by  POD  will  prevent  loss  of  the  graph  (or  other 
worksheet). 

All  of  the  plots  listed  on  the  Charts  menu  are  available  in  an  d  versus  a  analysis.  See  Section  2.1 
for  examples.  Only  the  Fit  plot,  Residuals  plot,  and  POD  plot  are  available  from  a  pass/fail 
analysis.  See  Section  2.2  for  examples. 


3.6  POD  Help  Menu 

The  Help  menu  contains  the  About  POD  menu  item.  Figure  27  presents  the  About  POD  window 
which  is  a  statement  of  proprietary  rights  and  limits  of  liability. 
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Figure  26. 


POD  Charts  Menu  Showing  All  Graphs  Selected  from  an  d  versus  a  Analysis 


About  POD 


y[-,pi  j  pod  Version  3.0  Copyright  (C)  2000 

pod| 


Copyright  2000  the  University  of  Dayton.  All  Rights  Reserved. 

This  material  may  be  reproduced  by  or  for  the  U.S.  Government  pursuant 
to  the  copyright  license  under  the  clause  at  DFARS  252.227-7013  (May  19871 

THIS  SOFTWARE  AND  ANY  ACCOMPANYING  DOCUMENTATION  IS  RELEASED  "AS  IS". 

THE  U.S.  GOVERNMENT  MAKES  NO  WARRANTY  OF  ANY  KIND,  EXPRESSED  OR  IMPLIED, 
INCLUDING,  WITHOUT  LIMITATION,  ANY  WARRANTIES  OF  MERCHANTABILITY  AND  FITNESS 
FOR  A  PARTICULAR  PURPOSE.  IN  NO  EVENT  WILL  THE  U.S.  GOVERNMENT  BE  LIABLE  FOR 
ANY  DAMAGES,  INCLUDING  ANY  LOST  PROFITS,  LOST  SAVINGS,  OR  OTHER  INCIDENTAL  OR 
CONSEQUENTIAL  DAMAGES  ARISING  OUT  OF  THE  USE,  OR  INABILITY  TO  USE  THIS  SOFTWARE 
OR  ANY  ACCOMPANYING  DOCUMENTATION,  EVEN  IF  INFORMED  IN  ADVANCE  OF  THE 
POSSIBILITY  OF  SUCH  DAMAGES. 


Figure  27.  About  POD  Window 
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